brotli-wasm
Part of HTTP Toolkit: powerful tools for building, testing & debugging HTTP(S)
A reliable compressor and decompressor for Brotli, supporting node & browsers via wasm
Brotli is available in modern Node (12+) but not older Node or browsers. With this package, you can immediately use it everywhere.
This package contains a tiny wrapper around the compress & decompress API of the Rust Brotli crate, compiled to wasm with just enough setup to make it easily usable from JavaScript.
This is battle-tested, in production use in both node & browsers as part of HTTP Toolkit, and includes automated build with node & browser tests to make sure.
Getting started
npm install brotli-wasm
You should be able to import this directly into Node, as normal, or in a browser using any bundler that supports ES modules & webassembly (e.g. Webpack v4 or v5, Vite, Rollup, and most others).
For each target (node.js, commonjs bundlers & ESM bundlers) this module exports a different WASM file & setup, with a slightly different entrypoint. These entrypoints all expose a consistent default-export API, in addition to some other exports that may vary (e.g. Node exposes the brotli methods synchronously, while browsers always require an await
due to WASM limitations).
In all builds (after waiting for the exported promise in browsers) the module exposes two core methods:
compress(Buffer, [options])
- compresses a buffer using Brotli, returning the compressed buffer. An optional options object can be provided. The only currently supported option is quality
: a number between 1 and 11.decompress(Buffer)
- decompresses a buffer using Brotli, returning the original raw data.
For advanced use data-streaming use cases, CompressStream
and DecompressStream
classes for streaming compression are also available. See the tests for example usage.
Usage
If you want to support node & browsers with the same code, you can use the await
browser-compatible form with the default export everywhere.
In node.js:
const brotli = require('brotli-wasm');
const compressedData = brotli.compress(Buffer.from('some input'));
const decompressedData = brotli.decompress(compressedData);
console.log(Buffer.from(decompressedData).toString('utf8'));
In browsers:
import brotliPromise from 'brotli-wasm';
const brotli = await brotliPromise;
const textEncoder = new TextEncoder();
const textDecoder = new TextDecoder();
const input = 'some input';
const uncompressedData = textEncoder.encode(input);
const compressedData = brotli.compress(uncompressedData);
const decompressedData = brotli.decompress(compressedData);
console.log(textDecoder.decode(decompressedData));
You can also load it from a CDN like so:
let brotli = await import("https://unpkg.com/brotli-wasm@1.3.1/index.web.js?module").then(m => m.default);
The package itself has no runtime dependencies, although if you prefer using Buffer
over using TextEncoder/TextDecoder
you may want a browser Buffer polyfill.
In browser with streams:
import brotliPromise from 'brotli-wasm';
const brotli = await brotliPromise;
const input = 'some input';
const inputStream = new ReadableStream({
start(controller) {
controller.enqueue(input);
controller.close();
}
});
const textEncoderStream = new TextEncoderStream();
const OUTPUT_SIZE = 100;
const compressStream = new brotli.CompressStream();
const compressionStream = new TransformStream({
transform(chunk, controller) {
let resultCode;
let inputOffset = 0;
do {
const input = chunk.slice(inputOffset);
const result = compressStream.compress(input, OUTPUT_SIZE);
controller.enqueue(result.buf);
resultCode = result.code;
inputOffset += result.input_offset;
} while (resultCode === brotli.BrotliStreamResultCode.NeedsMoreOutput);
if (resultCode !== brotli.BrotliStreamResultCode.NeedsMoreInput) {
controller.error(`Brotli compression failed when transforming with code ${resultCode}`);
}
},
flush(controller) {
let resultCode;
do {
const result = compressStream.compress(undefined, OUTPUT_SIZE);
controller.enqueue(result.buf);
resultCode = result.code;
} while (resultCode === brotli.BrotliStreamResultCode.NeedsMoreOutput)
if (resultCode !== brotli.BrotliStreamResultCode.ResultSuccess) {
controller.error(`Brotli compression failed when flushing with code ${resultCode}`);
}
controller.terminate();
}
});
const decompressStream = new brotli.DecompressStream();
const decompressionStream = new TransformStream({
transform(chunk, controller) {
let resultCode;
let inputOffset = 0;
do {
const input = chunk.slice(inputOffset);
const result = decompressStream.decompress(input, OUTPUT_SIZE);
controller.enqueue(result.buf);
resultCode = result.code;
inputOffset += result.input_offset;
} while (resultCode === brotli.BrotliStreamResultCode.NeedsMoreOutput);
if (
resultCode !== brotli.BrotliStreamResultCode.NeedsMoreInput &&
resultCode !== brotli.BrotliStreamResultCode.ResultSuccess
) {
controller.error(`Brotli decompression failed with code ${resultCode}`)
}
},
flush(controller) {
controller.terminate();
}
});
const textDecoderStream = new TextDecoderStream();
let output = '';
const outputStream = new WritableStream({
write(chunk) {
output += chunk;
}
});
await inputStream
.pipeThrough(textEncoderStream)
.pipeThrough(compressionStream)
.pipeThrough(decompressionStream)
.pipeThrough(textDecoderStream)
.pipeTo(outputStream);
console.log(output);
Note that TransformStream
has become available in all browsers as of mid-2022: https://caniuse.com/mdn-api_transformstream. It's also been available in Node.js (experimentally) since v16.5.0.
This is a simplified demo example - you may well want to tweak the specific stream buffer sizes for compression/decompression to your use case, to reuse buffers, or explore further optimizations if you're interested in these streaming use cases.
Alternatives
There's a few other packages that do similar things, but I found they were all unusable and/or unmaintained:
- brotli-dec-wasm - decompressor only, compiled from Rust just like this package, actively maintained, but no compressor available (by design). If you only need decompression, this package is a good choice.
- Brotli.js - hand-written JS decompressor that seems to work OK for most cases, but it crashes for some edge cases and the emscripten build of the compressor doesn't work in browsers at all. Last updated in 2017.
- wasm-brotli - Compiled from Rust like this package, includes decompressor & compressor, but requires a custom async wrapper for Webpack v4 usage and isn't usable at all in Webpack v5. Last updated in 2019.